TV animation interactively controlled by the viewer through input above a book page

ABSTRACT

A video system enables the operator to repeatedly touch a study object to change the action according to dramatics or game on a video display. 
     A educational embodiment enables that student to touch the page of a printed book and effect multiple types of video response, selected on page by a student, educating the student of the meaning of the point in the printed text or graphics touched. A similar embodiment provides a solid object the student touches to effect multiple animations. Multiplexed workbooks and monitors enable the teacher&#39;s writing or drawing in workbook to be displayed on all, or selected, monitors. Questioning by the animation or in the workbook is answered by touching designated areas in the multiplexed workbooks to select an answer, which is timed, graded, and recorded for the teacher&#39;s reference. The animation responds to the student answers. Animation location coordinates coded in the tracks of recorded video images are compared to player and instrument position coordinates obtained by ultrasonic detection.

CROSS REFERENCE TO RELATED APPLICATION

The application is a continuation-in-part of application Ser. No.07/228,946, filed Aug. 4, 1988, now abandoned.

BACKGROUND OF THE INVENTION

The present invention relates generally to interactive video systems andto improvements in U.S. Pats. Nos. 4,695,953 and 4,711,543 entitled TVAnimation Interactively Controlled by the Viewer. More particularly, thepresent invention relates to a method and apparatus wherein a viewerdynamically interacts with an animated video presentation to controlaction sequences and enable rapid and repeated switching of multipletracks of recorded data representing different actions while retaining acontinuous action sequence, and enable branching at the termination ofan action sequence by effecting track rewind.

U.S. Pat. No. 4,695,953 teaches a double-circuit video system that inone embodiment enables a player to repeatedly touch or hit an animatedcharacter during a continuous action scene as displayed on a projectionscreen or video monitor and thus change the action repeatedly. Anotherembodiment enables a player to swing a racket before the screen or videomonitor, hit the mid-air projected image of a perspective ball animatedaction, return the ball back to the animated character opponent, andplay a simulated game during which the player exercises the same skillsused to play the game simulated. An ultrasonic transducer in a playinginstrument and a microphone combination disposed on the video monitorface or at angles to the playing action permits the calculation of asignal representing the monitor contact position or the mid-air positionof the playing instrument action relating to the game. The signal thuscalculated is compared to a similar representation of the position ofthe character or object in the associated frame of the animationdisplayed by the video monitor and digitally coded on tape or disc.

U.S. Pat. No. 4,711,543 teaches a method of animation to provide smoothanimation displayed on the video monitor of a continuous action sceneduring rapid and repeated switching of multiple tracks of datarepresenting different animation action sequences.

While the interactive video systems disclosed by the above-cited U.S.patents admirably fulfill the roles of educational and amusementsystems, it it a primary objective of the present invention to extendthe capabilities and effectiveness of such systems.

SUMMARY OF THE INVENTION

The present invention provides an improvement of the interactive videosystem disclosed in U.S. Pat. Nos. 4,695,953 and 4,711,543, theimprovement comprising a combination of an ultrasonic transducer at thecontact tip of a hand-held pen indicator and an array of one or moremicrophones, the microphone array mounted in a frame cover adjusted in aregistered position over each printed page of a book permits acalculation of the position of the contact points of the pen to the bookpage. Such calculated positions are compared to coded animationproduction positions of said book page to effect multiple types ofresponse to pen indicated page position by switching and rewinding tomultiple tape or disk positions multiple tracks of recorded animationduring continuous action scenes displayed on a television screen orvideo monitor. Prerecorded or synthesized sound tracks associated withthe animated sequences may also be initiated to provide dialogue orexplanatory information associated with the pen indicated pagepositions. Printed typography, graphics, keyboards, circuits, flowcharts, diagrams, illustrations, printed music, or languages may betouched to effect multiple types of animated response of action to theinanimate page or analysis. The type of response to each indicated pageposition is selected by the student by touching printed indexes ordiagrams on the page.

In a first embodiment, a student working at a workbook utilizes the penindicator to either respond to monitor stimuli or to effect videomonitor response to the student's indicated book position stimuli. Themeaning of the student performance is determined and evaluated and theresponse is timed and scored. The student performance is then graded andprocessed for recording purposes.

A plurality of monitors and said book embodiments are operated in amultiplexed mode to serve a system comprising a number of students andan instructor. Means provided enable the instructor to write or draw insaid book embodiment and effect the display of this writing or drawingon all, or selected, student monitors. The instructor's writing ordrawing may appear overlaying displayed animation, a video reproductionof a selected book page, or appear on a blank screen. Means of audiocommunication between the teacher and all, or selected, students isprovided.

Means enable a student to touch printed keyboards on the pages of saidbook embodiment with an indicating instrument to operate a computer or acomputer with printer. Simplified keyboards consolidating multiple keycomputer functions enable a child or a person without any computerskills to operate a computer in a sophisticated manner with a relateddisplay of animation graphics.

In an alternative embodiment, positions on a three dimension object maybe contacted to effect the display of animation and associated soundexplaining and analyzing the indicated points or areas of said object inmultiple modes similar to the book embodiment.

In another embodiment comprising two ultrasonic transducer installed atthe barrel end and at the sighting positions of a rifle simulation and amicrophone array disposed at angles to said rifle produce doublereadouts of the mid-air positions of said transducers when the rifle isaimed at a point on a monitor screen and during the fraction of a secondwhen the player pulls the trigger. A calculation of the rifle positionand angle relative to the monitor screen produces screen target positiondigital equivalent which is compared to animation screen position codeddata to effect an animated reaction.

Other embodiments provide interactive dramatics between the animationand the player. The measurement of the player's speed, direction andarea of movement provides signals which initiate related reactions inthe animation. An embodiment provides modular robotic apparatus alsoreacting to the direction, speed, and movement area of the animation andthe player. Such robotics may also be manually operated.

The capabilities of the interactive video system disclosed in theabove-referenced patents are greatly increased by the incorporation of acomputer and printer. Coupled to the system secondary circuit thecomputer may control or supplement the system, and the printer caneither produce graphics or alphanumeric characters based on theoperator's touch of a printed page keyboard as described.

BRIEF DESCRIPTION OF THE DRAWING

The invention will be more fully comprehended from the followingdetailed description and accompanying drawing in which:

FIG. 1 is a schematic block diagram showing one embodiment of thepresent invention using a video-tape and a video-tape reader for theprimary circuit and a secondary circuit of digitally produced animationwhich may be used for scoring and diagram scenes.

FIG. 2 is a diagram in perspective of a book and half-frame microphonearray illustrating a second embodiment of the present invention.

FIG. 3 is a diagram in perspective illustrating the book and half-framemicrophone array of FIG. 2 assembled in accordance with the presentinvention.

FIG. 4 is a plan view of a page from the book of FIG. 2 illustrating apage number and video track preference coding system.

FIG. 5 is schematic diagram of a page of the workbook shown in FIGS. 2and 3 on which is printed a number of symbols representing key pads atpredetermined locations.

FIG. 6 is a conceptual block diagram illustrating the networking ofinstructor and student workstations in a multiplexed system constructedin accordance with the present invention.

FIG. 7 is a functional block diagram of an alternate circuitimplementing the systems shown in FIGS. 3, 4 and 5.

FIG. 8 is a functional block diagram implementing the instructor andstudent networked workstation system shown in FIG. 6.

FIG. 9 is a diagram in perspective showing the placement of discretepoint microphones in a triangular array with a transducer incorporatedin a hand-held indicating pen about a 3-dimensional study objectillustrating another embodiment of the present invention.

FIG. 10 is a functional block diagram of an alternative circuit of FIG.1 implementing the system shown in FIG. 9.

FIG. 11 is a diagram in perspective showing the placement of discretepoint microphones in a pyramid array with sonic tranducers installed ina rifle simulation before a video monitor illustrating anotherembodiment of the present invention.

FIG. 12 is a functional block diagram of an alternate circuit of FIG. 1implementing the system shown in FIG. 11.

FIG. 13 is a diagram in perspective showing the placement of discretepoint microphones in a pyramid array with a playing instrumentincorporating a sonic transducer before a video monitor illustratinganother embodiment of the present invention.

FIG. 14 is a diagram in perspective showing the placement of discretepoint microphones in a triangular array with a player and playinginstrument having transducer mounted thereon before a video monitorillustrating another embodiment of the present invention.

FIG. 15 is a functional block diagram of an alternate circuit of FIG. 1implementing the system shown in FIG. 16.

FIG. 16 is a conceptual diagram of the transmission of pulse waves to asingle-point receiver by an ultrasonic transponder.

FIG. 17 is a timing diagram of ultrasonic pulses transmitted andreceived.

FIG. 18 is a conceptual block diagram of an output terminal of thesystem shown in FIG. 1 that broadcasts digital control data to remotemodular apparatus.

FIG. 19 is a conceptual block diagram of an input terminal of remotemodular apparatus that receives transmissions from the system outputterminal shown in FIG. 18.

FIG. 20 is a functional block diagram of an alternate input circuit andindicator apparatus incorporating ultrasonic transducer to the systemshown in FIG. 1.

FIG. 21 diagrams a line projected from the centerpoint and normal to aprojected line between the transducer of the modular indicator apparatusshown in FIG. 20.

FIG. 22 diagrams a line projected through the transducer of the modularindicator apparatus shown in FIG. 20 intersecting a video monitorscreen.

FIG. 23 diagrams a mirror and lens system to display pictures in threedimensions.

FIG. 24 diagrams an equivalent to system diagrammed in FIG. 23.

FIG. 25 diagrams a three dimension viewing system using overlappingimages.

FIG. 26 diagrams rotating shutters used in system diagrammed in FIG. 25.

FIG. 27 shows a system to drive said shutters and generate a triggersignal to control the projection of left and right images.

FIG. 28 is a conceptual block diagram illustrating an opticalarrangement eliminating moving parts which employs crystals that can bemade opaque or transparent.

FIG. 29 is a functional block diagram illustrating a system to generatethe visual images of the system shown in FIG. 25.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to FIG. 1, there is shown a block diagram of the inventioneducational and amusement video system as a combination of two differenttypes of video-game circuits. The details and functions of the circuitsshown in FIG. 1 are more fully described in U.S. Pats. Nos. 4,695,953and 4,711,543 hereby incorporated by reference as if fully set forthherein. The primary circuit provides operational control of the systemand incorporates camera-originated motion pictures with audio assupplied in this embodiment by video tape 11 and video tape player 12. Asecondary circuit incorporates digitally produced or stored animation, auser controlled computer 58, a printer 59 and a multi-function outputterminal 60.

Referring to FIGS. 2 and 3, there is shown an embodiment of the presentinvention of a book 2 displaying printed graphics and typography onpages 7. The book pages 7 corresponds to the contact surface 7 shown inthe secondary circuit of FIG. 1. The book pages 7 are bound 20 to enablethe pages to lay in a vertical stack. Accordingly, the peg holes 11 and12 at the upper corners of pages 7 form a vertical cavity accepting theprotruding pegs 13 and 14 that are mounted on a half frame 3 containingtwo ultrasonic strip microphones 8 and 10 installed in each of the twomutually perpendicular sides. The half frame 3 strip microphones 8, 10corresponds to the vertical and horizontal strip microphones 8, 10 asshown in FIG. 1 and are coupled to logic circuit 21 by cables 181. Thehalf frame 3 is held by the operator with handle 9 and fitted into pegholes 11 and 12 on pages 7 in a registered position, as shown in FIG. 3.An indicating instruments or pen 6 incorporates an ultrasonic transducer5 at the ball point tip and is coupled to comparator circuit 18 (asshown in FIG. 1) by cable 191. Pen 6 also includes an internal switchwhich indicates whether or not the transducer tip 5 is in contact withthe surface of page 7. Pen 6 incorporates a pulse generator (not shown)or is coupled to pulse generator 22 (as shown in FIG. 1) by cable 191which causes the transducer 5 to produce repetitive ultrasonic impulses.The impulses can be produced by a ceramic transducer or high voltagesparking between two electrodes on the pen 6. The strip microphones 8and 10 together with transducer 5 in tip of pen 6 produce a signal whichis responsive to the placement of the head of pen 6 on the book page 7.This signal is converted to digital form and compared to a digitalreadout of coded page positions. This comparison is used to determine aspecific track of multiple tracks of animation, to determine a specifictape/disk position to rewind thereto according to the primary circuit ofFIG. 1, or to determine specific prerecorded digtial animation fordisplay or to produce digital animation according to the secondarycircuit of FIG. 1. The multiple tracks of animation of video tape ordisk 11 of FIG. 1, are camera-originated using the full theatrical-typeanimation production process or live action motion picture productionoptically retimed or analyzed for each frame's action and based on aframe exposure schedule sheet similar to an animator's exposure sheet,and whereon specific frames are digitally coded for animation imagescreen location coordinates or printed image page location coordinates.Said multiple tracks of motion pictures are displayed by video tape/diskplayer 12 and switched according to commands by processor 19 and cueingunit 53. Track rewind and reset is instructed by said processor 19 andcueing unit 53. The above described camera-originated animation may bereplaced by prerecorded digital animation obtained from ROM 27, ordigital animation produced to fit the requirements of the system bycartoon graphics generator 57.

The strip microphones 8 and 10, transducer 5, pen 6, and book page 7 ofFIGS. 2 and 3 are indicated by the same numbers in the block circuitdiagram of FIG. 1.

Referring to FIG. 1, the two strip micrphones 8 and 10 are responsive tothe ultrasonic pulses produced by instrument 6. These microphones, inconjunction with clock 23, counters 14 and 16, and logic circuitry 21,measure the time delay for airborne propagation between the radiation ofa pulse at instrument 6 and its arrival at the respective microphones 8and 10. Logic circuitry 21 combines the outputs from microphones 8 and10, pulse generator 22 and clock 23 to control counters 14 and 16 whichcount or measure the time delays. The measured time delays establish thecoordinates of the location of contact tip 5 on the surface of book page7 at the time of any particular pulse output. The application of thisknown ultrasonic method to detecting locations that are compared tocoded animation location to effect the course of animation is unique andunknown.

The animator's field positions of the graphics (FIGS. 2-3) displayed onbook page 7 is entered by the animator on specific frames of theproduction exposure sheet that instructs the exposure of those drawingsthat detail graphic positions to be contacted by instrument 6. Suchlocations per frame are converted to digital binary coordinates. Themotion picture frames are counted, specified, and used to enter a tableof values holding these graphic positions and vital data on the tape ordisc at the beginning of each page episode of the animation. All ofthese numbers are stored in the circuit RAM 51 of FIG. 1 and retrievedwhen applicable to the animation display. The frame number is recordedat intervals on the video tape track 11 and used as a track guide.Special track coding allows fast rewinds to specific frames located fromtrack signals. Referring to FIG. 1, the coded field positions with framenumbers are stored by player 12 from tape or disc 11 into RAM 51 at thebeginning of each page episode. This table of values in RAM 51 suppliespertinent data to logic circuitry 17 on time to be compared to thecoordinates of the pen 6 transducer tip 5 as it contacts the surface ofbook page 7.

The comparator circuit 18 of FIG. 1 compares the two field coordinatesderived from the digital readouts of counters 14 and 16 to the twocoordinates of the field and page 7 location of graphics locations fromlogic circuitry 17 and RAM 51. The comparator 18 instructs the processorcircuit 19 if specific field coordinates coincide. Based on this data anorder is dispatched to the system primary or secondary circuit toprovide specific animation. If such order is dispatched by processor 19to the primary circuit, the order is dispatched to video tape player 12which switches between a plurality of tracks, indicated in thisembodiment as 1,2,3, and 4, based on the program of the episode andtrack instructions from program memory cartridge 26, RAM 55, and cueing(cuing) unit 53. Or, the video tape player 12 may rewind to a positionin the tracks, based on said program. If such order is dispatched to thesecondary circuit, the processor 19 dispatches an order to cueing unit53 via vue table memory 54.

Referring to FIG. 1, the secondary circuit animation and graphics isgenerated by cartoon graphics generator 57 from digital data which maybe read along with digitalized audio from ROM memory 27 or othermass-storage device. Retrieval unit 52 is a conventional peripheralinput reader-controller which transfers into memory the digitally codedblocks of information obtained from ROM memory 27. This informationincludes control data which retrieval unit 52 stores into random accessmemory (RAM) 51 for use by dispatcher unit 19, and audio/or graphicsdata which unit 52 stores into RAM 55 for use by cueing unit 53. Thecontrol data includes cue commands and schedule commands. Cue commandsspecify short term operation during an interval of time, while schedulecommands represent longer term points of time, and form chains whichdefine and relate to alternate (multiple track) schedule. Dispatcher 19controls the course of the animation or graphics display, audio, andstores cue commands into cue table 54. Cueing unit 53 executes the cuecommands. Cueing unit 53 repeatedly scans cue table 54 to get commandstelling it what to do and the time it should be done. Dispatcher unit 19may request successive blocks of control information from retrieval unit52 and output into cue table memory 54 a schedule (called a cue table)of operations for cueing unit 53. Dispatcher 19 repeatedly updates thecue table schedule as the animation progresses. Dispatcher 19 processesthe various optional player input controls 29 which may input viaconventional video game playing instruments and stores the differentplayer commands into cue table 54.

As described, dispatcher 19 controls the course of the animation andstores cue commands into cue table 54. Cueing unit 53 executes the cuecommands at the times specified therein by conveying to the cartoongraphics generator circuit 57 blocks of binary-coded data previouslystored into RAM 55 by retrieval unit 52, and these blocks of data areused by the cartoon graphics generator 57 to generate animation frameswhich are then displayed on television monitor 25. Digital audio passesfrom ROM memory 27 through retrieval unit 52 to memory 55 to digital toanalog converter 56 and hence to system speaker 28. The binary codeddata stored into RAM 55 is reinforced by individual page or game datasuplied by program memory cartridge 26.

The primary multiple track video tape animation circuit, which isdiagrammed in FIG. 1, is operated and scheduled by the processordispatcher 19 which has control of the course of the camera-originatedanimation. The comparator 18 furnishes the results of the player'saction to the processor 19 which instructs the switching of multipletracks 1, 2, 3, and 4 with game or sequence position to video tapeplayer 12. At the termination of a game or sequence either random numbergenerator 20, player input 29, or the placement of instrument 6 on pagebook 7 specifies an episode to processor 19 which instructs tape player12 of the rewind.

The operation of both the primary and secondary animation circuits iscontrolled by processor 19 based on data from program memory 26 and ROMmemory 27 and the operation of the secondary circuit retrieval, cueing,and memory circuits as described above. As ordered by processor 19, thisinstruction is sent to tape player 12 or cartoon graphics generator 57by cueing unit 53. The digital memory cartridge 26 can be various typesof memory and may be plugged in to change the scenario, provide digitaldata for each book page 7, or to replace video/audio means 11 and 12 andsupply the coded animation used by the system.

The following invention embodiment distinguishes from prior art byoperating the above technology with the touch of a pen instrument 6 tothe printed page 7 while the parent inventions operate by touchinginstrument 6 to the face of the video monitor 24 or the mid-airmeasurement of the player's action in relation to projectedforeshortened perspective animations. The following invention embodimentoperates by comapring and relating the position of the pen instrument 6on book page 7 to digitaly coded location data recorded in theanimation.

Referring to FIG. 2, the drawing shows a book 2 with printed typography,graphics, or illustrations on pages 7, that is bound with a binding 20which allows the pages 7 to lay in a vertical stack, each page directlyover or under each other. The pages 7 may swivel to the back of the book2 and thus allow any page to be positioned as the top page. At the topcorners of the pages 7 circular holes are punched in the same positionsfor each page 7. When the pages are stacked vertically, as shown,cavities 11 and 12 are formed that accept protruding pegs 13 and 14 whenthe half-frame instrument 3 is placed by the student over pages 7 asindicated in FIG. 3. Accordingly, the strip microphones 8 and 10installed in the mutually perpendicular frame sides of frame 3 arepositioned in a registered position relative the top page 7 as shown inFIG. 3. A handle 9 is grasped by the student when positioning frame 3and pegs 13 and 14 into cavities 11 and 12 over pages 7. The studentplaces a transducer 5 installed in the tip of pen 6 to the surface ofpages 7 and by this process activates a switch indicating contact andduration of contact to the system which functions as previouslydescribed. Alternatively, the transducer 5 is installed in the fingertipof a glove worn by the user or student and the selected page 7 positionidentified to the system by bringing the glove fingertip into closeproximity or touching the selected location on the page 7.

The embodiment illustrated in FIGS. 2 and 3 incorporates pen 6 and frame3 wired 191 and 181, respectively, to the circuit of FIG. 1. As analternative a remote control embodiment of pen 6 and frame 3 eliminateswiring 181 and 191 to pen 6. Pen 6 and frame 3 incorporate batterydriven modules with crystal controlled synchronized time bases. Penmodule 6 incorporates an ultrasonic oscillator producing pulses, andframe module 3 incorproates a radio transmitter sending the pulse datafrom microphone 8 and 9 to a receiver in the remote input circuits 61via antenna 63 in the circuit as shown in FIG. 1.

The means of identifying the book page 7 number to the invention systemare displayed in FIG. 4. A keyboard 15 comprises number locations storedin the system memory and allocated to an area on page 7, but notprinted. The page number digit area is printed on the page as indicatedin area 16 for digit 2, the first digit number of the page, in this case"26". The second number is located in area 17 for digit 6. The studenttouches pen tip 5 to areas 16 and 17 to register "26" to the system. Analternative miniature version of the above reduces the keyboard 15 to aconfiguration of dots, the applicable dots of which are printed on thepage border as indicated by 22, 23, and 24 to register positions inrecorded miniature keyboards of the page numbers. The miniaturekeyboards are enlarged to identify the book to the system. The studenttouches pen tip 5 to these dots 22, 23, and 24.

The invention provides multiple animation responses to a singleindicated page 7 position, or chain of positions. The student selectsthe type of response from a listing of responses printed on the page 7.Each listing may be preceded by an outlined area which may be a square21 as shown in FIG. 4. Such areas 21 may be interspersed in sentences ofthe text to register a selected type of response. The student touchesarea 21 with pen tip 6 to select.

Multiple tracks of animation are produced as responses for the subjectmatter of the printed pages 7. The animated data can also include soundtrack data to provide a verbal description associated with the printedmatter on the page 7 at the selected positions. The sound track data mayalso provide background sound such as music associated with selectedpage positions, either in addition to descriptive matter or instead of.In one embodiment, the video portion of the animated tracks may bedisabled or animation tracks providing only sound track data may beutilized to provide audio animation and description only of the printedmatter at the selected page position. This animation production is timedand scheduled for frame exposure by the animator on an exposure sheet.The page location coordinates of areas effecting response when touchedby transducer 5 of pen 6 are calculated on the exposure sheet to becoded into the finished animation production at the start of everyscene. The said coordinates are based on the measurement of the standardfield which is 8.75 inches high and 12 inches wide. The embodiment shownin FIGS. 2 and 3 turns this standard field into a field 12 inches highand 8.75 inches wide. A reduction in said 12/8.75 ratio may be made, asto a printed page area of 9.6 inches high and 7 inches wide, which fitsa page 11 inches high and 8.5 inches wide, the typewriter paper size.Accordingly, the animator's coded field location coordinates would becalculated as percentages of the field height and width from half-frame3. Thus the center of the printed field would be 50% South and 50% East.An alternate embodiment of book 2 and frame 3 reverses the height andwidth measurement, and conforms to the standard animation field.

In addition to the response described above by camera-originated tracksof full animation, the invention provides other means of response usingpre-recorded digital animation as described in the functions of thesecondary circuit and voice response from a vocoder or similar chip togenerate voice (analog signals) from digital code, computer digitalanimation generated at the time, and pre-recorded and generatedanimation at the same time. Each of the above means for esponse has adistinct use and advantage.

The student has the ability to operate the system by touching keys of akeyboard printed on the book page. Referring also to FIGS. 1 and 5,there is shown a number of locations preprinted on page 7 within thearea bounded by frame 3 representing keyboard pads. The pad locationsare coded and input to the system from tape or disk 11, or fromcartridge 26. The student positions pen indicator 6 in close proximityto or touches the page contacting the surface on one of the pads, pad Bfor example, with pen point 5. This indication together with thehorizontal 10 and vertical 8 sensors allows the system to register thekey stroke as pad B in the multiple choices of pad A through pad E.Alternative, the user could answer YES or No. For a person that does notread, keypads can be color coded as shown. The ability to touch keypadson the printed keyboard to effect the proper response in a typewriter ordesk-top computer and printer is provided by utilizing indicating pen orinstrument 6 with the printed representation of an entire keyboard. Penpoint 5 contact determines the page 7 coordinates of the key pad touchedwhich are matched to a coded table of values for all the key padpositions to effect a digital command to an associated desk top computerwhich preferably matches the same digital data generated by the computerkeyboard keys when touched to initiate the corresponding computerdisplay or printer action.

A typewriter or computer keyboard printed in an instruction book may betouched to control a computer, a printer, and the TV monitor display ofthe system according to the keys touched. An illustration of a musicalinstrument keyboard, as a piano, may be touched to play the sound of thekeys and display the notes on a musical scale. Touching a note in amusical scale printed on the book page plays the sound of the note anddisplays the corresponding key being touched on a piano or formed by thehands on a guitar or other instrument. Musical symbols are related tosounds, and music is related to symbols. Touching the letters of theunknown word in a reading primer with pen 6, the illiterate or childhears the sounds of the word, the whole word sound, the visibleappearance of the human mouth in action, as each sound and word isspoken and pronounced in synchronization with the position of thetransducer 5. Handwriting of the printed word is written insynchronization with the sound and animated formation of the printedletters. Thus the child learns to associate the unknown printed lettersymbols and the handwriting thereof with his spoken vocabulary and theformation of words with his mouth towards learning to read, write, andpronounce. A single letter or word may be touched repeatedly by thetransducer 5 to afford the beginner an exploration of all the elementsof the unknown printed word on his own terms, or comprehension. Foreignwords and sentences as printed in the book are pronounced with Englishtranslation as the English equivalent is animated on the screen withactions and pictures of the meaning according to the placement of pen 6on page 7. And, thus English printed words may be translated to aforeign language. Multiple animations may explain and depict multipleanalyses of a single sentence. Accordingly, the student may choose, bytouching a printed key space on the page, to hear the sentencepronounced as the animation depicts the letters building into words, tohear each sound pronounced and combined into each word, to have thesentence meaning explained, to have a single word defined as adictionary, to have the grammer of the sentence analyzed, to have thesentence translated to another language, or to have the meaning of aparagraph explained and illustrated in action. A keyboard table ofspeech sound letters arranged according to the science of phonetics maybe touched by the student with pen 6 to teach the letter symbolsaccording to the manner and place of their related sound formation. Bytouching the known word sounds as (w-u-n-s) both the sounds of theword,(wuns), and the irregular spelling, "once", may appear on themonitor with audio. Maps and charts are animated when touched by pen 6.Details of geography are explained on the television monitor whenindicated by the touch of pen 6 to a printed map. The movement andaction of electricity in circuit diagrams is animated and explained ineach detail that is indicated by the student. The details of instructionmanuals are converted to animated actions that may be speeded or slowed,or branch to multiple other actions during continuous action.Perspectives of graphics may change in animation, and objects may changeto other objects in an animated metamorphosis. Thus the invention buildsin the printed word or picture all the features of the art of animation,the computer, the printer, and the television, to be activated by thestudent, by himself, as he touches the book page 7 with pen 6.

While the preferred embodiment of the book 2 shown in FIGS. 2, 3 and 4is described as incorporating both video and audio, an alternateembodiment of the book system using audio alone may be used to teacheffectively. Touching unknown printed words that are in his speakingvocabulary and shown in a book 2, the beginner or illiterate hears thesounds of the printed word symbols and thus is able to break thesound/symbol code. The alternate embodiment uses a coded audio tape ordisk, and the system operates otherwise as herein described.

A workbook embodiment of the above book invention provides differentmodes. In one mode the student answers questions in the workbookpertaining to the animation display on the monitor. The animationdisplay may then offer an instructive response to the student's answers.In another mode the student watches the monitor and makes inputs to theworkbook in response to monitor questions by positioning the penindicator to locations on the workbook printed matter of text, charts,graphics, illustrations, diagrams, or any form of printed imagery. Thesystem may then switch to multiple tracks of animation in response tothe indications on the workbook by the student. In a combination of theabove modes the student responds to workbook questions by activating avideo display response and responds to monitor questions in theworkbook. Audio stimuli is also used that is appropriate to augment thevisual to enrich the instruction or simulation or realism.

The student working at a workbook 2 using a pointer 6 to indicatepositions on the page 7 is able to graphically define the display shownon the monitor 24 in multiple modes. Assuming a student is studying thecircuit of FIG. 1 which is printed on a workbook page 7, by touchinglogic circuitry 21 with the indicator pen 6, the student has graphicallydefined and effected a video display showing and explaining the animateddetails of flow charts, circuits, and function of logic circuitry 21.The student may, also, define the type of analysis by touching squares21 printed on the workbook page 7 at preselected locations withdescriptive graphics and text of the types of video/audio response. Thecoordinates of the location of square 21 on the page 7 constitute a codewhich is also coded in the animation production or supplied by programmemory cartridge 26 for access from memory storage RAM 51 to be comparedto said digital coordinates determined by the student graphical inputterminal and from counters 14-16 of FIG. 1. In other modes involving thedefinition of action and the timing thereof, the student may indicateverbs whose meaning is animated in action on the display. Indicating theprinted verb "run" defines the displayed run action. The handwriting ofscript writing is animated in the proper pen progression to teach thestudent how to write. The script forms in synchronization with thestudent's indicating the definition of a printed word and accompanyingaudio. Or, as the student touches the notes of music printed in bars onpage 7, as the notes are being played and visually performed on thevideo monitor 24, the timing and accuracy of the student's indicationsmay be graded for synchronization by data sent by cuing unit 53 tocomputer 58 and printer 59 for processing. In another mode the studenttouches the printed text, graph, diagram, chart, keyboard of answers, orillustration preprinted on page 7 that is related to the subject beingdiscussed or displayed on monitor 24. Then, the accuracy, timingsynchronization, and action of the student's indications are scored andgraded from data sent by cuing unit 53 to computer 58.

The embodiment may be used for training or games in addition to thetesting and workbook uses. The system provides means determining themeaning of the student response, evaluating the student performance,timing the response, and recording such findings. The performanceevaluation results output may be transmitted by the computer 58 to aninstructor at a remote or central location via output terminal 60.

Several modes of operation involve scoring. First, the student'sresponse can be used to determine the next action in self-pacedlearning. Second, examinations can be given and graded following alearning experience. Both require comparison of the user's response witha stored correct answer. In working with the book 2 and monitor 24, avariety of input options would be available, including multiple choiceand keyboard responses. The correct response would come from an inputvia Program Memory Cartridge 26, Video Tape 11, or Graphics Generator57. The coordinates of the correct (and allowed wrong) locations arestored in RAM 55, and the computer 58 would compare the response to theallowed locations to determine if a response is right or wrong. If anumber or alphabetical response is permitted as an answer, the userwould use a printed keyboard in the workbook 2 to indicate the response.The user's response can be used to either control the sequence ofactions or generate a grade based on performance. The results may beprinted on printer 59 or sent to an instructor by computer 58.Performance evaluation could also be based on time taken to respond. Atthe start (turning a page, for example) the time from clock 23 can bestored in a register in computer 58. When a response is received, thetime taken is readily available by subtracting the register value fromthe current time from clock 23 (i.e., an electronic stop watch).

Referring now also to FIG. 6, a plurality of monitors 24 and workbookembodiments as described above may be operated in a multiplexed modewith a master monitor and workbook to allow an instructor to serve anumber of students with considerable economy. FIG. 6 shows a conceptualblock diagram of a networked system providing a master or instructorstation 61 multiplexed with a number of slave or student stations 63 vianetwork interface circuitry 65. The instructor station 61 comprises thesystem shown in FIG. 1 including a terminal or keyboard 67 to allow aninstructor to provide input commands and other instructions to thesystem. The instructor station is coupled to the network interfacecircuitry 65 via the system output terminal 60. Each of the studentstations 63 comprises a remote monitor and workbook system (as describedwith FIGS. 2-5) coupled back to the instructor station 61 and the systemvia the network interface 65. In addition to the indicator pen 6, eachstudent station 63 may also comprise a terminal or keyboard 69 toprovide alternate input means. The instructor station monitor can beswitched between individual student stations 63 to allow an instructorto interface with or monitor any student individually, a selected groupor all of the students simultaneously.

The instructor utilizes the indicator pen 6 to write, or drag orotherwise mark on the pages 7 of the workbook 2 as a blackboard. Theinstructor utilizes keyboard 67 or other means to select on whichstudent's or students' monitor the blackboard display would bedisplayed. Student inputs and responses light displays on the masterterminal or monitor to indicate to the instructor that a studentresponses is waiting for the instructor attention. Selection of theparticular student station displays the student's notes or otherresponse on the instructor monitor. In a multiplexed mode, theinstructor's input via the pen 6 and workbook 2 and/or selected animateddisplays associated with selected page 7 locations are simultaneouslydisplayed on all of the student station 63 monitors. Student responsescomprising animated displays provided by selected ones of multipletracks of animation data associated with student indicated page 7locations are displayed on the student monitor when selected by theinstructor. The master/slave system may be implemented locally, or thestudent stations may be remote from the instructor station.

Referring to FIG. 1, the instructor uses the pen instrument 6 to writeor draw graphics on book page 7. The pen 6 includes a writing tip 5.Described means for producing digital signals representative of the twocoordinates of the location of pen tip 5 on the surface of book page 7produce a chain of such position coordinates for the writing or drawingby said pen tip 5. In a well known manner, this digital representationis then used as an address in a RAM organized in a raster representationof the screen of the monitor. This is read out by a scanning system fora line at a time to produce the video signal driving the monitor in aconventional manner. The instructor's writing may thus appear on a solidcolor or blank monitor screen, on an operating animation display, or ona still video reproduction as of the printed type or graphics on thepages 7 of said book or workbook.

Referring now also to FIG. 7, a functional block diagram of a circuit inaccordance with the present invention is shown which provides thecapability to input video display data via the indicator pen 6 incombination with the page 7 and book frame 3 for display either alone orin combination with animated display data. A normal video signal frommultiple sources, for example, video tape player 12 is coupled tomonitor 25 via a mixer 62. This signal would be in some standard formatsuch as NTSC (National Television Standards Committee). To write with oroverwrite display graphics, or indicate a pointer or cursor incombination with the animated display, a second user input video signalinput via indicator pen 6 and strip microphones 8 and 10 is combinedwith the normal signal in the mixer 62. This could be combined in atleast three modes as determined by a control signal and the mixer 62 asfollows: show normal video only; i.e., prerecorded or stored animation;show user input video only; show both signal combined; and show digital(foreground) superimposed over the animated display; the animateddisplay may or may not be deleted in regions where the user input videoimage is displayed. The user input video image signal is generated bythe user in digital form and coupled to mixer 62 via digital-to-analogconvertor (DAC) 56. A graphical input terminal as shown in FIG. 7comprising a surface 7 corresponding to a page 7, a writing instrumentor pen 6 data responsive to movements of the writing pen 6 at thesurface 7 representing the location of pen 6 coupled to computer 58. Thewriting instrument 6 includes a transducer at the writing tip 5 and afelt marker, pencil lead, or the like. Instrument 6 also includes aninternal switch which indicates whether or not the writing tip 5 is incontact with the surface 7. The computer 58 generates command signalsrepresenting the coordinate data for the writing instrument 6 referencedto the surface or page 7 boundaries accessing raster data stored in RAM51. A digital video image is generated by RAM 51 and coupled to DAC 56to provide an analog video signal to mixer 62. As previously described,strip microphones 8 and 10 are responsive to the ultrasonic pulsesproduced by writing instrument 6. Microphones 8 and 10, clock 23,counters 14 and 16, and logic circuitry 21 measure the time delay forairborne propagation of pulse from transducer 5 to microphones 8 and 10to provide the raw coordinate data to the computer 58. Alternatively amouse may be used inputting data directly to the computer 58.

Referring now also to FIG. 8, a functional block diagram of analternative circuit to that shown in FIG. 7 implementing themaster/slave system shown in FIG. 6 is provided. A synthesized videosignal derived from user inputs via indicating instrument 6, synthesizedby the secondary circuit shown in FIG. 1 and output from the DAC 56, forexample, is coupled to a video signal processor 818, 819 on line 811,813, respectively, for an individual station. Additional user inputgenerated by computer 821, 823 is combined with the video signal onlines 811, 813, respectively, and coupled via switching controllers 825,827 to the appropriate station mixer circuits 829, 831 where it iscombined with selected animated video from an animation video input 812such as the video tape player 12 or plug-in cartridge 26 of FIG. 1. Thecombined video signal is coupled to the station display monitor 815,817. Any number of stations may be added on to the system thus providinga networked system in which any stations may be coupled to one or moreof the other stations on the network. Utilizing switching circuits 825,827 which also perform multiplexing functions provide the capability fora master station to control displays on all other stationssimultaneously.

Referring now also to FIG. 9, an alternate embodiment in accordance withthe present invention utilizes a 3-dimensional solid object which may beany form of mechanism or sculpture and is shown as cube 25. One or morearrays comprising three point or discrete microphones 8, 9 and 10 arepositioned at angles around said solid object study 25 sufficient toencompass the surface area and enable a student to study any point onthe object 25 as described in the above book embodiment. For example,the study object 25 may represent a globe of the earth with the3-dimensional coordinates of selected points or the globe associatedwith animated video display and sound tracks providing informationconcerning the country or region surrounding the selected point. Analternate embodiment provides for rotation of the study object 25 toregistered fixed positions, thereby requiring only one array of threediscrete microphones as shown in FIG. 9. Each fixed stop position forthe study object has a separate table of location values defined. FIG. 9illustrates an installation of single discrete point microphones 8, 9,and 10 before side 24 of object 25. The student indicates the point onthe study object 25 to be studied with indicator 6, as the pointindicator pen 6 tip 5 touches the side of object 24.

Referring now also to FIG. 10, a functional block diagram showscircuitry which replaces the similar circuits and apparatus of FIG. 1.Using the three triangular formation variable measurements of microphone10 to 8, 5 to 9, and 5 to 10, 3-dimensional location coordinates of thepoints on object 25 touched by indicator instrument 6 are determined andinteractively related to the animation displayed on the monitor 25. Akeyboard module comprising the player input control section 29 (as shownin FIG. 1) is used by the student to select the category of animationresponse. The uses include training and simulation systems. The solidobjects studied may be machinery, motor vehicles, globe maps, weaponry,medical physiology models, taxidermy, instruments, tools, farmequipment, and the interior circuitry of electronic devices astelevisions and computers. The animation may display a cross-section ofthe interior action of such studies as a human heart or automobilecaburetor. Multiple objects may be studied as a plurality of mounted,connected, or associated objects. The student may repeatedly reviewanimated instruction according to his own comprehension development. Theembodiment creates another dimension of multiple animated actionanalysis for the classroom, sales display, vocational training, or anyother form of studied object education. An alternate embodiment ofindicating instrument 6 would be a glove with a transducer 5 installedat the index finger tip and may use portable module circuitry,crystal-timed with a power source.

Referring now again to FIG. 1, the system shown includes computer 58coupled to printer 59. The computer 58 is coupled to the host system viathe host processor 19, the retrieval unit 52, the curing unit 53 and RAM55. The computer 58 receives and processes commands and otherinformation and provides system commands to the host processor andsystem output to the printer 59 and to the system output terminal 60 viathe cuing unit 53. The computer 58 also responds to user inputs via theplayer input control block 29 and the host processor 19. Printer 59produces graphics or typing according to the contact of the pentransducer 6 to the printed computer or typewriter keyboards describedin the above book embodiment. Many types, sizes configurations, andcolors of keyboards printed in the said book may operate the computer 58and printer 59 when the keyboards are touched by said pen 6. Or, printedtype describing computer functions may activate the function describedof the computer when the type is touched by said pen 6.

Computer programs controlled by simple descriptive keyboards or graphicsas described may be operated by a layman without any knowledge of acomputer operational language such as MS DOS. Accordingly, the inventionallows the beginner or layman to do everything a computer user needs todo without getting into the technical aspects of computer operation.Computer program software is incorporated in the book keyboardembodiment process allowing the layman to concentrate on computerfunction concepts presented by multiple types of keyboards as described.The layman may use a plurality of keyboards during a single computerfunction. Other functions of the computer may operate using graphicsymbols as a keyboard. Flow charts may serve as computer keyboards. Thusthe invention simplifies using a computer by enabling the layman tooperate a computer without using the standard keyboard and computerprogram software directly. But, the layman does use the standardkeyboard and the various accessory program software indirectly.

Further, multiple tracks of animation may elaborate and explain saidprinted keyboards to the layman in every detail that the laymanindicates in said book with said pen. Such animated response would beseparated from the above described computer response.

Referring to FIG. 11, there is shown a drawing of a simulated rifleaimed at the face of a video monitor 24, or at a motion pictureprojection screen. When the rifle is aimed and triggered the inventioncalculates the precise point on the video monitor 24, or projectionscreen, whereto the rifle barrel 6 to 5 is aimed. Accordingly, the rifleembodiment is unique by instantly producing the coordinates of the pointof aim on the screen. The usual firing of a missile, emission of lightrays, or the manipulation of video monitor circuitry to effect a slidingcursor into the target all become extraneous and not intrinsic to thesimulation process. Thus the gun embodiment may be used on largeprojection screens and incorporate large weaponry. The use of largeprojection screens also heightens the sense of reality by enlarging thetarget area. Transducers 6 and 5 working with point microphones 8, 9 and10 permit the accurate determination of position and aiming direction ofthe rifle 7 relative to the screen 24. The calculation of the aimingspot may be done using various sytems. The rifle may be aimed atspecific points on the screen in the set-up mode to implement theinitial condition measurements. Using rectangular coordinates, thepositions may be calculated by ratios. As an example, Distance 33-31 isequal to Distance 22-21 multiplied by the ratio of Distance 6-33 toDistance 6-22. The various ratios may be organized to use only thedirectly measured values converted into their vertical and horizontalcomponents. If the rifle 7 is fixed relative to the screen, these ratiosmay be fixed and the computation simplified by storing thepre-calculated fixed terms. (Calculation can avoid division and usefaster multiplication if instead of dividing by some number, itsreciprocal is calculated, stored, and used as a multiplier.)

In this manner, all the other required distances to locate the aimingspot on the screen 24 can be calculated. Accordingly, there are improvedelegant methods that use less computer power that are an advance of thesimple geometric method as follows: Ultransonic trandsucers 6 and 5 areinstalled at the extremities of the rifle barrel at the sightingposition 6 and at the point of the rifle barrel 5. These transducersalternatingly emit impulses to the point microphones 8, 9, and 10 whichare installed at equal distances from the base points 11 and 12 ofmonitor 25 on lines at right angles to said base line 11-12 and at thecenter of base line 11-12. A readout of said impulses by microphones 8,9, 10 establishes the coordinates of transducers 6 and 5. The verticalelevation line of transducer 6 to floor point 16 is calculated forlength and position of floor point 16. The length of line 16-17,intersecting base line 11-12 at right angles, is calculated to obtainthe length of comparable parallel line 6 to 33. The vertical elevationline of transducer 5 to floor point 26 is calculated for length andposition of floor point 2, 6. The vertical difference of point 6 topoint 5 establishes point 21. The distance from transducer 5 verticalline base 26 to transducer 6 vertical elevation line base 16 to 17 lineis calculated to obtain comparable parallel distance 5 to 23. Thedistance of 6 to 21 is calculated from the right angle triangle 6-5-21using the known measurements of 6 to 5 and 5 to 21. The known distance 6to 21 using the known measurements of 6 to 5 and 5 to 21. The knowndistance 6 to 21 is, then, used with the known distance 21 to 22 in theright angle triangle 6-21-22 to establish the distance of 6 to 22. Thedistance of 6 to 33 is divided by the distance of 6 to 22, and theresulting divisor is multiplied by the distance 21 to 22 to establishthe distance 33 to 31. The said divisor, is then, multiplied by thedistance of 22 to 23 to establish the distance of 33 to 35. The point onthe video monitor screen 24, or projection screen, at which the rifle 7is aimed is established by a vertical measurement of the known 33 to 35distance to point 30, and the horizontal measurement of the known 33 to31 distance to point 30. Said aimed "hit" position 30 is then comparedto the target position of the animated object of character displayed bythe video monitor 25 and digitally coded into the video tape or disc.The course of the animation is then changed according to the accuracy ofthe rifleman. The animation, if "hit", reacts with sound duringcontinuous action. Other animation embodiments may include targetscoring, instruction, or game graphics. The rifle simulation effect isenhanced by mechanical spring or electronic means causing the rifle to"kick" on being triggered to a degree simulating reality.

Referring now also to FIG. 12, a functional block diagram of the rifleembodiment circuit which may be incorporated in the system doublecircuit of FIG. 1 is shown. The rifle 7 is a portable battery operatedultrasonic generator module. The generator module includes ultrasonictransducers 5 and 6 for emitting ultrasonic pulses. The transducers aredriven by an ultrasonic oscillator keyed to produce pulses of ultrasonicenergy at a fixed repetitive rate. A crystal controlled time basesynchronizes the frequency, pulse width, and repetitive rate of thepulses. Transducers 5 and 6 each produce a pulse signal differing inpulse width and frequency. Accordingly rifle module 7 producesalternating pulses of two different types, one type emitted bytransducer 5, and another type emitted by transducer 6. Pointmicrophones 8, 9, and 10 detect the ultrasonic energy emitted bygenerator module rifle 7. Logic circuitry 21 recognizes the designationof each alternating type pulse by the pulse width and frequency.Accordingly, logic circuitry 21 divides the three pulses of themicrophones 8, 9, and 10 to become six pulse signals to measure thethree coordinates from each transducer 5 and 6. Counters 14, 15, 16, 36,37, 38 convert the pulse data to digital form to be compared bycomparator 18 to digitally coded animation location data.

A stereo visual perception embodiment of the present invention extendsthe scope and effectiveness of the inventions and embodiments thereofdescribed hereinabove and in the referenced U.S. patents related toanimation interactively controlled by the viewer. Although foreshortenedperspective animation admirably propjects an image to a mid-air positionbefore the monitor or projection screen, and although the speed of suchprojection renders such projected images undiscernible in animation, asthey are in reality, a hundred miles per hour for a baseball pitcher'sserve, a stereo visual perception process embodiment of the presentinvention provides slow moving objects and permanent three dimensionprojections before the screen to appear to the operator, or player, asreality, and thus heighten the simulation illusion. The preferredcurrent three dimensions (3-D) image scheme uses two pictures sent inalternating fields of a video (TV) signal. The viewer, or player, uses apair of glasses with electric crystal "shutters" that allows only oneeye to see at a time. The regular video (TV) picture is sent at 60frames a second with alternating frames showing alternating lines(interlaced). Thus a half of the picture is sent in 1/60th of a secondand complete pictures at 1/30th a second. By going to a higher signalbandwidth, the 3-D system sends one picture for one eye with nointerlace in 1/160th of a second and both eyes see a complete 3-Dpicture 30 times a second.

There are many ways to display and view three-dimensional video. FIG. 23shows a mirror and lens system 230 arranged to give separate views foreach eye 231, 233. If properly adjusted with visual images 235, 237 foreach eye 231, 233, respectively, that are different perspective, thehuman brain believes the scene shown in the images in three-dimensional.The equivalent apparatus arrangement is shown in FIG. 24. The images235, 237 can be taken with a camera that is shifted to the eye positionsin an arrangement like FIG. 24 or the images can be computed or drawn bythe animator. FIG. 24 works with filters 239 as shown with the opticalpaths including separate lenses 241 separated for the two eyes 231, 233.If the left eye image 237 is green and the left filter green while theright eye image 235 and filter for the right eye 231 are red, the twoimages 235, 237 can be projected overlapped on the screen and yet seenseparately.

The above arrangement can be used as shown in FIG. 25 with the right andleft images 235, 237 overlapped on screen 238 provided that some methodis used to allow only one eye 231, 233 at a time to see the overlappedimage. For example, two rotating shutters 243, 245 as shown in FIG. 26rotating at 24 or 30 times a second alternately allow one eye 231, 233,respectively, to view the overlapped image 235, 237 at a time. (Theshutters could have two opaque and two transparent segments each and berotated at half speed). FIG. 27 shows a conceptual diagram of a systemto drive the shutters 245, 243 and generate a trigger signal to controlthe projection of the left and right images 237, 235. Two projectors canbe used with shutters, one for each eye image. The trigger then can beused to keep the eye shutters properly synchronized.

Moving parts can be eliminated by optical arrangements employingcrystals that can be made opaque or transparent based on an appliedelectrical signal. An alternate method could employ a crystal thatchanges (rotates) the plane of polarization based on an applied voltage.When a crystal of this type is used in combination with a polarizingfilter, the light path can be open or blocked. FIG. 28 shows aconceptual block diagram for a circuit to control voltage-controlledcrystal 247, 249 characteristics.

The un-energized system is configurated with the polarizer and crystalpolarization at some angle, 45 to 90 degrees, for example, to eachother. When a voltage signal is applied to the crystal, the polarizationof the crystal is rotated to align with the polarization of thepolarizer which allows light to pass through. The images 235, 237 canalso be polarized to provide additional control of light paths. FIG. 29shows a conceptual block diagram of a system to generate the visualimages 235, 237 displayed on screen 238 as shown in FIG. 25.

Referring to FIGS. 10 and 13 an embodiment of ultrasonic positiondetection incorporationg 3-D stereo visual perception is shown. Atransducer 5 is installed in a baseball bat 6, as shown, or in thesocket of a catcher's glove. Three point microphones installed at points8, 9, and 10 together with transducer 5 creates pulse readouts measuredas shown in FIGS. 10 and 13 that establishes the mid-air position oftransducer 5, and this position is interactively related to the 3-Danimation mid-air position of, in this case, a baseball pitch. Theplayer may also see and catch a slow moving thrown ball using acatcher's glove with transducer 5 installed. The catch is registered bythe system when the animation ball position compares to the glovetransducer 5 mid-air position. Mechanical or electronic means create anagitation in the socket of the glove to simulate reality with the systemsound effects. Another system embodiment enables the player to swing aracket before the screen or video monitor, hit the mid-air stereo visual3-D projected image of a perspective ball animated action, return theball back to the animated character opponent in stereo 3-D perspectiveaction, and play a simulated game exercising the same skills as the gamesimulated. Such games, as tennis, ping-pong, badminton are thus playedon animated stereo 3-D courts that change stereo 3-D perspectives as theaction progresses from side to side of the court. Multiple stereo 3-Dembodiments include the rifle described.

The process of producing animation provides means to provide two imagesof animated scenes calculated for each eye to create a stereo perceptionillusion of art work that is of only two dimensions. The cel animationmay be photographed on the animation camera in two positions that widenas the foreshortened perspective action approaches the viewer. The righteye sees a cel drawing adjusted on the moving top or bottom pegs of theanimation camera to a position toward the left, while the left eye seesanother positioning of the same cel drawing 10 adjusted on the pegs to aposition toward the right. And, two different animation drawings may beanimated, in another means, to create different perspectives for eacheye. Of course, such a process requires double the amount of animationdrawings. Another means to create a stereo perception illusion is in thebackground art. The animation camera incorporates multiple moving topand bottom pegs. These pegs carrying sections of backgrounds may move atdifferent speeds (spacing) according to the depth of perspective foreach level. A tree in the background foreground would move on widermoves per frame than a grove of trees in the background distance. Thismovement is also augmented by slightly different peg positions of thetree and grove for each eye. If the animation camera is moved up or downfrom the compound art work, the resulting truck in or back is reinforcedby separating or joining background levels, as trees, foreground objectson the sides of the scene, and these moves may also be calculated foreach eye.

We have discovered embodiments extending the scope and effectiveness ofthe inventions and embodiments thereof described hereinabove and in thereferenced U.S. patents by improving and advancing interactive dramaticsbetween the player and the animated characters and graphics. Theimproved reactions of the animated characters and objects are based onthe player's actions, speed of the player's actions and positions of theplayer and playing instrument, all obtained by transducer/microphonereadouts.

Referring now to FIGS. 14 and 15, a system and associated circuitry forimproved interactive dramatics between a player and displayed animatedaction sequences is shown. In FIG. 14 a transducer 6 is mounted on thebody of player 4 and a transducer 5 is mounted in a glove or playinginstrument 7. The player moves in an area whose base on the floor islocated by the corners at positions 41-42-43-44 before video monitor 25.Transducers 5 and 6 working with an array of discrete point microphones8, 9 and 10 provide accurate determination of the three coordinates ofthe mid-air position of player transducer 6 and playing instrumenttransducer 5. As shown in FIG. 15 a portable battery operated modulethat is mounted on the player 4 includes a pulse generator 22 and anultrasonic oscillator 40 keyed to produce pulses of ultrasonic energy ata fixed repetitive rate. A crystal controlled time base 39 (synchronizedwith time base 23) synchronizes the frequency, pulse width, andrepetitive rate of the pulses. Transducers 5 and 6 each produce a pulsesignal differing in pulse width and frequency. Accordingly the portablemodule produces alternating pulses of two different types, one typeemitted by transducer 5, and another type emitted by transducer 6.Discrete point microphones 8, 9, and 10 detect the ultrasonic energyemitted by transducers 5 and 6. Logic circuitry 21 recognizes thedesignation of each alternating type pulse by the pulse width andfrequency. Accordingly, logic circuitry 21 divides the pulse signalsfrom the microphones 8, 9, and 10 to become six pulse signals to measurethe three coordinates for each transducer 5 and 6 positions. Counters14, 15, 16, 36, 37, 38 convert the pulse data to digital form to becompared by comparator 18 to digitally coded animation location datacorresponding to the animated display on the monitor 25 at a (time)frame.

The means of measuring the speed of the player transducer 6 and theplaying instrument transducer 5 is based on measuring the velocity ofthe transducers 5, 6 relative to the receiving microphones 8, 9 and 10.

Various well-known means of ultrasonic measurement may be used accordingto the game needs, distances, and vector geometry. FIG. 16 shows anultrasonic transponder XMIT 1 which transmits audio waves or pulses 2 inresponse to an electrical signal. These are received by a single-pointmicrophone REC 3. FIG. 17 shows these pulses transmitted in position 4and received at position 5. To avoid problems with ambiguity, thetransmit pulses, must be spaced in time farther apart than the longestdistance of interest to be measured. Alternatively, the receivingmicrophone REC 3 may be gated. A clock or digital counter is initiatedby the transmit pulse. When the leading edge of the received signal isdetected, the counter is stopped. The time "tx" can be converted todistance when the velocity of sound in the medium is used as a scalefactor. This process is simple if it occurs in one plane, if pointreceivers and transmitters are used and there are no reflections. Anadditional measurement to determine relative motion between thetransmitter and the receiver can be made using the Doppler effect. TheDoppler effect is determined by measuring the frequency of the receivedpulses and comparing this frequency to the frequency of the transmittedpulses. The change in frequency between the transmitted pulse and thereceived pulse is a measure of the velocity of the transmitter relativeto the receiver, and is a factor in determining the velocities of thetransmitter 5 mounted on the playing instrument 7 and the transmitter 6mounted on the player's body 4.

There are two ways to determine velocity from the invention system: A)change in position divided by time interval between two measurements;and B) the Doppler effect. As the measurement does not have to beprecise for game playing, it will be easier to use the Doppler effect.The Doppler effect provides a measurement only of the component ofvelocity in the direction of a line between the object or transmitterand the sensor. To calculate the total velocity, three sensors in threecoordinates are used and the vectors added. For the speed measurementsof the system, it may be sufficient to use just the measurement of onecoordinate.

The movement of the player transducer 6 and the player instrumenttransducer 5 shown FIGS. 14 and 15 is in paths of action and directionsthat are calculated from the progressing frames of positions determinedby the coordinates of the transducers 5 and 6. A three dimensionalplaying area before monitor 25 of FIG. 14 has a base 41-42-43-44 and atop 45-46-47-48 and may be divided into area zones. Also a zone curingarea may be calculated around the animated character or object displayedon the face of the monitor screen 24 or at the projected mid-airposition of animated foreshortened perspective actions. Accordingly, thevelocity, mid-air position, monitor screen 24 contact position, path ofaction, direction of path of action, zone position relative to the videomonitor, and zone position relative to the displayed animated characterare position measurements of the player 4 and playing instrument 7 thatare compared to position and schedule data relative to the animationdisplay which is coded in the animation production or supplied byprogram memory cartridge 26 for access from memory storage RAM 51 ofFIG. 1 to generate a sequence of digital commands to direct theanimation display. This embodiment of the present invention increasesthe degree of animated character dramatic sensitivity to the playeractions and thus creates a sense of reality. Accordingly, if the playerapproaches the screen, the animated characters may react and run away orattack. A player moving away from the screen may cause animatedcharacters to appear. If the player moves in any direction the animatedcharacter may appear to follow. A quick move by the player may frightenthe animated characters. Any move by the player may cause a reaction inthe animation. The movement of the player into area zones related to themonitor, or related to the animation character and display, may cause areaction, or schedule of reactions, by the animated character.

The embodiment described extends the scope of game action. Tennis orPing-Pong games as contemplated in U.S. Pat. No. 4,695,953 are improvedby the realistic and more sensitive reactions of the animated characteropponent to the player moves, direction, area position, and speed.

Referring now to FIGS. 1, 18 and 19, the system of FIG. 1 provides thecapability of controlling the action of remote vehicles and otherapparatus based on feedback of actions of such remote apparatus. FIG. 18shows a functional block diagram wherein output terminal 60 receivescommands and other data for remote modular apparatus or robotics andtransmits the data in digital or analog form via transmitter 61 to suchapparatus. As shown in FIG. 19, remote modular apparatus includes inputcircuitry for receiving commands and other data via a receiver 62 whichis controlled by a crystal controlled time base 67. Command and datasignals are coupled through interface 69 to allow the use of multiplemodules to processor 68 within the remote apparatus to operate themodule. Alternatively, output terminal 60 may be directly coupled tointerface 69 for relaying command data to interface 69 and hence toprocessor 68. Commands and other data transmitted to processor 68 aregenerated by the system circuit shown in FIG. 1 and coupled to theoutput terminal 60 via cuing unit 53.

An embodiment of a modular apparatus which may be controlled by datafrom cuing unit 53, is diagrammed as apparatus 7 in FIGS. 20, 21 and 22.Apparatus 7 operates in a coded area before video monitor 25 as shown inFIG. 14. Transducers 5 and 6 are shown installed at extremities ofapparatus 7. Transducers 5 and 6 emit ultrasonic pulses provided bypulse generator 22 and ultrasonic oscillator driver 40 according to acrystal controlled time base 39. As previously described, microphones 8,9, and 10 receive pulses and the position coordinates are measured inthe same manner as in the description of the operation of FIG. 12. As wehave described the system function in relation to FIGS. 11, 12, 14 and15, the determination of the coordinates of the moving transducers 5 and6 supply the mid-air position, the coded zone area position, the speedand direction of movement of the transducers, and the point on the videoscreen 24 of FIG. 11 that is the intersection of a projection of a linebetween the transducers and the screen 24, all of which may be comparedto animation display coded data, as described, to effect both the videodisplay multiple track switching and the behavior or restriction of thebehavior of apparatus 7 while being manually operated by the trainee orplayer. The scope of the combinations of the above factors encompassesmultiple simulated training or entertainment apparatus controlled beforethe video screen.

Referring to FIG. 20 the apparatus 7 operating as a module before atheatrical size video screen 24 may be embodied as a simulated vehicle 7driven by the operator. Transducers 5 and 6 are mounted on the sides ofvehicle to register tilt. The vehicle 7 is mounted on a mechanismcontrolled by animation coded data transmitted as shown in FIGS. 18 and19, and as described above. The vehicle 7 also moves from side to sideaccording to the inputs generated steering by the operator. The positionof the vehicle 7 before the video screen 24 is continuously establishedby command digital coordinates from coded data determining the distancesof transducer 6 to microphones 8, 9 and 10 and transducer 5 tomicrophones 8, 9 and 10 to adjust the vehicle 7 to simulate the wheelsrunning on animated racetrack displayed on the screen 24. Accordingly,the vehicle elevates and tilts to fit the displayed animated racetrackand the operator steers the vehicle from side to side on the displayedanimated racetrack.

The operator steering of the simulated vehicle can effect the animateddisplay related to the steering in various embodiments. The operatorsteering of the vehicle is free and unrestricted to steer the vehicleoff the racetrack at either side. Simulating reality, the event of suchan accident effects both the animation and vehicle. The intrusion of thevehicle into coded areas before the screen triggers an animated reactionincluding sound effects and the violent shaking of the vehicle.

Another embodiment provides animated obstacles on the oncomingforeshortened perspective animated racetrack that may animate out of theway, or they may not. Defensive driving is required by the driver. Ifthe coordinates of the location of the vehicle 7 match the codedcoordinates of the displayed obstacle, at frames of the animationdepicting collision, the system effects track switching as previouslydescribed.

Referring to FIG. 20, the apparatus vehicle 7 may be steered by theoperator to aim at a projected point on the video screen 24, and thuseffect the animation aimed at, or effect the animation based on anindication related to the point of aim. FIG. 21 shows a line 1 projectedthrough transducer locations 5 and 6. A line 2 is projected from thecenter of the distance between transducers 5 and 6, normal to line 1,forward to the video screen surface 24, at point of aim 3. The screencoordinates of the point of aim 3 may be determined by a method usingthe distance 5-6 as the fixed distance of the hypotenuse of right angletriangles. The method is illustrated in FIGS. 11 and 12 and described inan embodiment of object 7 as a rifle. The added calculation of theprojection line 2 at right angles to the projection line used in therifle embodiment is incorporated in the process. An embodiment examplemay provide the selection of one of a number of forking routes of theoncoming roadway at a branching point in the animated roadway of adesignated frame wherefrom multiple tracks originate depicting multipleroutes of the separating road. The animation track of the branchingroute aimed at by the vehicle is selected by the system for display.

The determination of the point of aim 3 by simulation vehicle 7 on videoscreen 24 is a process instrumental to an embodiment using a simulatedgas pedal effecting the speed of the oncoming perspective foreshortenedanimated roadway giving the illusion of speed to the vehicle 7. As thegas pedal is pushed to the points of accruement per frame, the mechanismof the pedal effects a truck to a smaller field (in animation productionterms) using circuitry for selecting a smaller TV field fraction of theanimation production field, then centering and expanding the selectedpicture so as to occupy the entire raster of the tube. The centers ofthe successive reduced fields are an aligned series aimed at point 3 onscreen 24.

FIG. 22 diagrams object 7 of FIGS. 11 and 12 as a cannon mounted on atank simulation, as the forward guns in an aircraft, or other weaponrysimulation. The simulated craft and weapon is operated to aim at a point3 on the video screen 24. When triggered the point of aim 3 isdetermined as described in relation to the rifle embodiment diagrammedin FIGS. 11 and 12. If the aim is correct the displayed animationdepicts a simulation of reality at the point of aim 3. When triggeredthe vehicle 7 reacts with sound effects simulating reality. The processsaves ammunition and affords infinite simulated firings and practice toimprove skills.

In another example, an amusement park embodiment of tennis or ping ponggames as contemplated in U.S. Pat. No. 4,695,953 is played before atheatrical video screen. A moving floor similar to moving belt walksinstalled in airports may be moved to match corresponding animation panbackground moves displayed in the animation. The animation is coded toproduce digital commands by the system moving the floor to fit the gameor dramatic action of the display. The output terminal 60 of FIG. 18transmits control commands to the modular apparatus operating the movingfloor based on the location of the player and the direction, path, andspeed of the game action effecting animation pan background moves.

Toy and amusement devices operated by remote modular apparatus asdiagrammed in FIGS. 18 and 19 may be activated and controlled byanimation coded data to react to the displayed animation, and may bemanually operated simultaneously. Another embodiment may be a toy dogthat progresses according to operator control, and also barks or acts inreaction to the displayed animation.

Toys that are confined to an area or that are stationary in locationalthough moving may be installed adjoining the video screen in locationswhose coordinates in relation to microphones 8, 9 and 10 of FIG. 14 arecoded in the system to be compared to coordinates of the player 4 andthus control the toy. Other embodiments may be toys that react to playeraction and animation action simultaneously. For example, a parrot mayspeak to the player when approached or comment and squawk about theanimation display. Puppets activated by mechanical means may function asan audience reacting to specific player action or animation action orboth. Action based on a comparison of player coordinates toenvironmental coordinates may be activated by the location of the playerand the direction, path, and speed of the player's actions.

The present invention contemplates scoring, grading the operator'saction and response, timing game action, and recording data by remotemodular apparatus. The operational data is relayed from the system bycuing unit 53 of FIG. 1 to output terminal 60 and hence wired ortransmitted to remote modular apparatus as diagrammed in FIGS. 18 and19. There are numerous other embodiments that include scoring and gametiming displays and clock devices that show the progress of mulitplegames, such as tennis. Remote switching devices may turn off or on thelights in a room to match game explosion displays, game disasters, ordarken the lights for dramatic effect. Educational embodiments includeclocks and displays timing and grading the student response.

While the motion picture display of the present invention has beendescribed as animation, it is understood that live-action motionpictures may be manufactured, produced, shifted by optical printing,retimed and analyzed for the coordinates of the positions of movingobjects to parallel the animation production methods of the invention,and thus be operated by the invention apparatus to perform the uniquefunctions herein described. Live-action motion picture frames may beenlarged and printed on animation size paper; then, moving charactersmay be cut out, put on animation cells (of clear plastic sheets), placedon animation or photographic backgrounds and photographed on motionpicture film according to the animator's exposure sheet to parallel theanimation method and production coding process of the invention.Computer generated images may be organized and analyzed to determine thecoordinates of moving objects to parallel the invention process.

While the invention has been described with reference to preferred meansand embodiments, it is understood by those skilled in the art that othermeans of position detection, such as a light pen, mouse or joy-stickvarious modifications, alternate arrangements, different circuits andapparatus that parallel the animation method and production coding andcoordinate matching process of the invention are within the scope of theinvention performing the unique functions of the invention. Therefore,animation descriptions, circuits specified, and apparatus specifiedshould not be construed as limiting the scope of the invention.

What is claimed is:
 1. A video system providing repeated switching ofmultiple tracks of animation and rewinding to specific locationsresponsive to positions indicated by a user on pages having printedimages of a book having a registration portion, said video systemincluding a primary circuit comprising:storage means for storingmultiple tracks of animated motion picture sequences, said animatedmotion picture sequences including coded frames for track switching andcoded page location coordinates corresponding to preselected printedimages on the pages of the book; video switching means coupled to saidstorage means for switching said multiple tracks of animated motionpicture sequences for rewinding thereof to specific locations for trackselection; a first user input terminal, including--an indicatinginstrument having a transducer installed at a tip thereof to producesignals, a portable book frame having detection means, installed inmutually perpendicular sides thereof, for detecting the signals fromsaid transducer, the registration portion of the book positioning saidportable book frame in a registered position over a selected page of thebook, means for producing digital position signals representative ofcoordinates of a placement location of said indicating instrument withrespect to the printed images on the selected page for selecting aprinted image, and means for identifying the book and a page number forsaid video system and selecting a type of video response, said videoswitching means responsive to said digital position signals to retrievea selected animated motion picture sequence corresponding to the printedimage selected by the digital position signals; and display meanscoupled to said video switching means for display of the selectedanimated motion picture sequence.
 2. A video system as in claim 1wherein said animated motion picture sequences include animated scenesrelated to said preselected printed images.
 3. A video system as inclaim 2 wherein said animated motion picture sequences include soundtrack data providing audio response associated with said animated scenesand related to said preselected printed images, said display meansincluding audio means for providing audio corresponding to said soundtrack data.
 4. A video system as in claim 3 further comprising audioselection means coupled to said storage means and said video switchingmeans and responsive to a user input for selecting an audio onlyresponse to placement of said indicating instrument.
 5. A video systemas in claim 1 further comprising processor means coupled to said firstuser input terminal and said video switching means for processing saiddigital position signals and controlling selection, switching andrewinding of said multiple tracks of animated motion picture sequences.6. A video system as in claim 5, further including a secondary circuitcomprising:a second user input terminal coupled to said processor meansproviding user input instructions for control of said video system;graphics generating means for generating predetermined animated videoimage sequences from stored graphics image data for display on saiddisplay means; memory means, coupled to said processor means and saidgraphics generating means, for storing said graphics image data, controldata and instructions including cuing, scheduling and video processingcommands; and cuing means coupled to said processor means, said memorymeans and said graphics generating means for executing cuing commandsfrom a cue table for control of said graphics generating means providinganimated graphics frames for display on said display means in responseto inputs from said first and second user input terminals.
 7. A videosystem as in claim 6 further including memory cartridge input meanscoupled to said processor means and to said display means for receivingplug-in cartridge memory for storing and providing video image dataincluding audio data associated with preselected pages of preselectedbooks.
 8. A video system as in claim 5 further comprising computermeans, coupled to said processor means and including an input terminal,for inputting user commands and instructions, said computer meansresponsive to said user input commands and instructions to control saidvideo system via said processor means.
 9. A video system as in claim 8further comprising output means coupled to said computer means andcontrolled by said computer means for outputting system data to theuser.
 10. A video system as in claim 8,wherein the printed images onselected book pages comprise a diagrammatical representation of acomputer keyboard, and wherein said multiple tracks of animationincluding keyboard data representative of visual images of the computerkeyboard, said keyboard data including related instructions associatedwith each of said keyboard keys, said computer means responsive to therelated instructions associated with user selected keys on said selectedbook pages.
 11. A video system as in claim 1,wherein said video systemis used in conjunction with workbooks, each workbook having aregistration portion and pages with printed images corresponding toanimated motion picture sequences, wherein said video system furthercomprises:a master station producing master station command signals andcomprising said first user input terminal and said display means, saiddisplay means including--a display, and sonic detection means disposedalong at least two mutually perpendicular edges of said display, saidsonic detection means including means for generating position signalsindicative of a placement location of said indicating instrument withrespect to selected visual images of a currently displayed animatedmotion picture sequence, said video switching means responsive to saidposition signals for retrieving a predetermined animated motion picturesequence corresponding to said selected visual images; a system outputterminal; a plurality of slave stations coupled to said system outputterminal and in multiplexed fashion with said master station, each slavestation comprising:a user input terminal including a moveable indicatingpen having a sonic transducer installed at a tip thereof to producesonic signals, a portable work book frame having sonic detection means,mounted in mutually perpendicular edges thereof, for detecting the sonicsignals from said moveable indicating pen, the registration portion ofeach workbook positioning said portable workbook frame in a registeredposition over a selected workbook page means for generating workbookposition signals indicative of a placement location of said movableindicating pen with respect to the printed images on said selectedworkbook page for indicating a selected printed image, means foridentifying the workbook and the selected workbook page for said videosystem and selecting a type of video response, said video switchingmeans responsive to said workbook position signals to retrieve aselected animated motion picture sequence corresponding to said selectedprinted image, and video monitor means coupled to said video switchingmeans for display of said selected animated motion picture sequence; andnetworking means for coupling said master station to said plurality ofslave stations in a switched and multiplexed fashion, including controlmeans responsive to the master station command signals for controllingthe display on said master station display means and on said videomonitor means of said slave stations.
 12. A video system as in claim 11wherein said master station includes graphic means, coupled to saidmaster station input means, said display means and said networking meansand responsive to said digital position signals, for displaying useroriginated graphics on said display means and on selected video monitormeans of said slave stations.
 13. A video system as in claim 12 furtherincluding communications means providing audio communications between afirst user located at said master station and second users located atselected ones of said plurality of slave stations.
 14. A video systemproviding repeated switching of multiple tracks of animation andrewinding to specific locations responsive to positions indicated by auser on a surface of a study object, said video system including aprimary circuit comprising:storage means for storing multiple tracks ofanimated motion picture sequences, said animated motion picturesequences including coded frames for track switching and coded pagelocation coordinates corresponding to preselected positions on thesurface of the study object; video switching means coupled to saidstorage means for switching said multiple tracks for rewinding tospecific locations for track selection; a user input terminal,including--a moveable indicating instrument having a transducerinstalled at a tip thereof to produce signals, detection means, disposedin fixed relationship to said study object, for detecting the signalsfrom said transducer, and means for producing position signalsrepresentative of coordinates of a placement location of said moveableindicating instrument with respect to preselected positions on thesurface of the study object, said video switching means responsive tosaid position signals for retrieving said animated motion picturesequences corresponding to a selected one of said preselected positionson the surface of the study object; and display means coupled to saidvideo switching means for display of said selected animated motionpicture sequence.
 15. A video system as in claim 14 wherein saidanimated motion picture sequences include animated scenes related to thestudy object and corresponding to the preselected positions on thesurface of the study object.
 16. A video system as in claim 14,whereinsaid animated motion picture sequences include sound track dataproviding audio response associated with said animated scenes andrelated to the study object and corresponding to the preselectedpositions on the surface of the study object, and wherein said displaymeans includes audio means for providing audio corresponding to saidsound track data.
 17. A video system as in claim 14 further comprisingrotating means coupled to the study object for rotating the study objectabout a predetermined axis of revolution to a one of a plurality ofpredetermined fixed positions, said multiple tracks of animated motionpicture sequences including a plurality of predefined animated motionpicture sequences, each associated with one of said plurality ofpredetermined fixed positions.
 18. A video system as in claim 1 whereinsaid indicating instrument comprises a glove worn by the user, saidtransducer mounted in a fingertip of said glove.